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state of the Art in Rehabilitative Audiology: Speech Intellibigility 
Invited Presentation, Convention of the American Speech and Hearing 
Association I Las Vegas, 1974 

Martin C. Schultz 

My remarks today deal with prospective directions for reseeurch in speech 
recognition and comprehension of the hearing impaired. Some of these suggestions 
come from surveying and critiquing current research trends, others arise out of 
one or another kind of model cibout how the hearing impaired client hears speech, 
and may not reflect an extrapolation from studies presently known to be under 
way. My suggestions are not necessarily mutually consistent nor supportive of 
the thoughts of our two learned colleagues. They are offered in a spirit of 
energizing discussion and mutual sharing of ways to attack these difficult human 
problems. 

Let me begin with a theoretical perspective. 

As we are all aware. Shannon presented us with a system for quantifying 
the uncertainty of messages so that we could apply the mathematics of probability 
to ongoing signals, as they were transmitted and received. This approach gave 
rise to much research about Markov processes in signal generation and got us 
involved in questions like. If I know the first three words, can I guess the 
fourth? or. How much assistance do I get in synthesizing a total signal when 
I receive parts of it with reasonable clarity but not other parts? And, as 
we are also all aware, the research didn't give us anything like the insight or 
productivity we had early expected so that by the time Chomsky*type rule systems 
put a stranglehold on probabilistic approaches to speech, they may have strangled 
what was already a corpse. 

But my point is that we considered Shannon's work and applied it in much the 
same ways as he did. Those particular applications were to prove quite successful 
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to the coDununications engineer but they were not and are not the application 
we need. 

If we stop to consider where probabilities are important to us, I think 
they are useful in two places. The first is in the choice of message by the 
talker • 

SLIDE I, PLEASE 

Z\*i» is an expanded Shannon model and let me point out the areas of 
expansion • First/ the motivation for the communication is the desire on the part 
of the talker to energize some particular response from the listener. He has 
available to himself an infinity of messages he can choose but, somehow the 
desired response serves as a filter to extract only that set of messages that 
have a probability greater than zero of motivating the desired response. At 
the other end of the slide, we have a listener, who is much like the talker in 
that he can receive and understand almost an infinity of messages, but in this 
situation from that talker and whatever he perceives as the talker's motives/ 
there is only a finite set of messages he is prepared to receive. The talker 
settles on a single message that, presumably, maximizes the prob2U3ility of 
engendering the attitude or motivating the behavior the talker desires. The 
hearing impaired listener has to get that message/ however it is encoded/ so 
understanding message choice behavior / rather than the area of message encoding 
is of importance to us. The other locus of importance for probability applica- 
tions is in the signal reception and interpretation task of the listener. 

SLIDE 2/ PLEASE 

Clearly the listener can use his knowledge of all components of the communi- 
cation in trying to receive and interpret the signal successfully. By saying 
all components what I convey in a formal sense is that he processes three kinds 
of information from momory: he ipust deal with parameters of possible signals / 
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probabilities of these signals as interpreted, and the utilities of these 
interpretations (their values and costs). In an informal sense, the listener 
can use his knowledge of the language including the particular social dialect 
being used, the probabilities of sounds and syllables and words, the cues of 
place, situation, social milieu and so forth toward establishing what the intent 
of the talker might be. All of these are aspects of signal reception and 
interpretation to which probabilitistic considerations are properly applied. 

SLIDE OFF 

Wh^ should we deal with probabilities? Because I think it might be fruitful to 
consider that the normally hearing individual, on the average, can cake advantage 
of so much redundancy in the signal and situation that seldom has he difficulty 
interpreting signals reasonably. The hearing impaired peison, on the other hand, 
runs into repeated difficulty, seems to have to rely on progressively less 
reliable and/or progressively lower probabilities, and, not only is he frequently 
wrong but it progressively upsets his confidence in his om abilities and he 
establishes a system of self-fulfilling prophesies about failure. In other mrds, 
the hearing impaired individual develops different probabilities than does the 
normal and fixes different values and costs to his decisions and there are two 
implications from that. First, in any research we do on the normally hearing 
with the idea that we can extrapolate to the hearing impaired, we must be very 
careful that these differences in probabilities and in payoffs will not be 
signilicantly involved. When such differences exist, the extrapolation clearly 
is at risk. The second implication is that we really don't know very much 
about how individual hearing impaired people hear speech so let me pursue that 
for a minute. 

When we view hearing rehabilitation as occurring at the periphery, when 
we view the ear as a mechanical-acoustical instrument, when wc view the hearing 



aid as an electroacoustical instruinent, we are emphasizing the physical 
world. We are looking at the signal and the distortions imposed upon it. For 
some selected purposes that orientation may be adequate. But there is major 
work to be done in the framework of the human being as information processor 
who brings much more than a distorted signal reception system to his task. 
He nas linguistic, social and speaker-oriented expectations, he has cues at 
the phonetic level, the syllable level, the phrase level etc. with which he 
tackles interpretation of the distorted signal. Some of these cues are of 
assistance, some of them are no longer operative, and we must presume that some 
are now systematically misleading and therefore probably very highly dis- 
ruptive. Running monosyllabic word tests or even isolated sentence tests 
will not give insight into many of the client's information processing and 
social processing problems, when you do that kind of extrapolating without 
taking those different probabilities and utilities into account, operationally 
you have decided they have no importance to you though they may be more 
important for understanding the client than anything you actually test about 
his communication. That's one focus for research, let's go on to another. 

One needed research azimuth deals with the client's trustmrtliiness in 
hearing aid quality judgments* The problem arises in finding effective and 
economical hearing aid fitting procedures. Such procedures must balance the 
current desires of the client for what he thinks he remembers clarity to 
be like, in the framework of the acoustical characteristics of the combination 
of low fidelity instrument and low fidelity ear, against his long run needs 
for effective speech cue information. It seems unclear that the client is 
the best immediate judge of what aid will, in the long run, give him best per- 
formance. If we could get him interacting with a computer, then the most 
effective hearing aid selection might turn out to be strongly influenced by 



a plotting of the learning curves he would generate m discrimination and 
recognition tasks which included trial-by-trial machine feedback. It seems 
intuitively that an aid yielding steeper slopes holds nvore promise, whether 
the patient is immediately more satisfied with the sound familiarity of that 
aid or not 

Associated with this last question, if one believes in seme kind of 
"wired-in" theory of speech recognition, then a major area of investigation, 
barely tapped with the hearing impaired population, involves the question of 
how much plasticity has the hearing impaired individual or indeed even the 
normal for learning new cues, for erasing useless or confusing cues? What 
kind of adjustment time does he require for changing cue systems, if he can 
do that# and how can we help him in this task? The question of cue plasticity, 
or one's ability to alter response strategics, is an area for future work* 
it is obvious that much of this work can be done with the normally hearing 
but a great deal of it will require hearing impaired subjects. I would add 
one caution before moving to another direction for research. 

If there is a special portion of the nervous system that processes speech 
and if it is true that that processing differs from processing of non-speech 
signals, then the experiments examining plasticity of the speech recognition 
system had better restrict themselves to the use of speech as the signals in 
the experiments. And I would remind you that if we are forced to conclude 
that the hearing impaired subject hais only very limited plasticity — that 
somehow he is constrained to follow the lead of whatever cues he docs get, 
misleading or not, ambiguating or not, then auditory rehabilitation will have 
to take on a much more synthetically-oriented rather than analytically-oriented 
character. Wc will, in this case^ have to do the research to learn how to train 
our client to assume postures and poses that will cause tlie talker to speak 



louder or articulate more precisely. We will have to do the experimental 
work to learn how to train our client to manipulate a variety of non-verbal 
communication configurations. Equally important, we will have to help 
him enhance the sensitivity of his expectations on the talker. 

Another topic* How does one write the equation between auditory and 
visual input for the hearing impaired? We know that, except under difficult 
circumstances, the normally hearing person doesn't have to see the talker 
at all* As his auditory syst experiences progressively more difficulty, 
his visual system makes a progressively larger contribution. But should we 
automatically take the view that the auditory system alone is to be maximized 
in hearing aid fitting? Is it appropriate witliout specifically researching 
the question, to accept the orientation that a hearing aid is to maximize 
speech recognition in the auditory domain? Should one even care which modality 
contributed what portion of the aided plus visual speech intelligence score? 
Perhaps a hearing aid most emphasizing cues that unambiguate the lip-reading 
contribution would be worth thinking about* I would remind you that one cost 
of selecting hearing aids based strictly on the combination score would be 
that we would lose the control on speaker variance now coming from standardized 
recorded tests. But an aging population and the likelihood that progressively 
more often hearing aids will be purchased by third parties requires a re-examin- 
ation of the argument* 

For those of you who might not have read the very powerful review article 
on tactile displays of speech by Kirman in Psych. Bull tor 1973, I would say 
that it may be time to rethink the potential contribution of the tactile 
sensory system to the processing of auditory information — partially speech 
recognition, but also binaural signal selection and background squelch, perhaps 
vocal pitch information, perhaps speech timing information and so forth* 



In this day of large scale, t me-shared comi, iters , I would like to see 
some materials developed that would be programmable for auditor y training 
so that a client could simply sit down at a console, call for a program or 
programs that would force him to work on discrimination in a mode that would 
give him feedback trial by trial, at his own pace, without occupying a clini- 
cian. The computer could also give the clinician the learning curve of the 
client so tiiat decisions could be forthcoming about when to shift materials, 
or to shift rehabilitation focus, etc. The computer could as easily deliver 
a short passage and inquire £Qx>ut topic area, or what emotion was being 
expressed or whatever. That is to say, it could as easily work on a synthetic 
approach as on an analytic. But we need such programs. 

You might consider further that, when your client is seated in front of 
the computer terminal, his earphones should be wired out-of-phase so as to 
maximize the contribution that the asymmetry of the waveform makes to his 
understanding . 

Let me go back to the psycholinguistic position that language is wired-in 
to the left hemisphere. It seems to t^e that one implication, worthy of study 
but also of consideration for your patients right now, in that the ear of 
choice for a hearing aid should be the right one. We all know that I cannot 
use this rule unequivocally without some specific research directed to the 
question but I raise the point even though it has likely struck many of you 
before. The recent study of Frederikscn and his colleagues in Scandinavian 
Audiology about the inability of some significant proportion of patients with 
presbycusic hearing loss to wear binaural amplification raises the same ques- 
tion about which should be the ear of choice for a single aid. The wired-in 
model suggests that it frequently should be the right ear even when that one 
seems to be the poorer. How much poorer and whether or not we should revise 
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the general guidelines we use for making that decision are ->w reopened 
as empirical questions. 

Here is still another tack for hearing ai& appraisal not currently in 
the literature — no better but different. I would like to see someone 
evaluate a two-level strategy where the first level defines reasonable goals 
for the patient in making the decision that he will opt for an aid or aids 
or not. I suggest something like a multiple regression on age, magnitude 
of threshold shift and perhaps its slope, percentage discrimination loss, 
education, ability to withstand adverse S/N conditions, proximity to 
binaurality, job and social demands, emd perhaps other considerations. 
All of this is done without an aid. The second part, done with an aid or 
aids, is a more refined analysis involving aided loss, discrimination or 
recognition scores with progressively larger multisyllabic units so thai 
coarticulation properties and the subject's expectations can be expected 
progressively to obscure phoneme discriminability , and ability to withstand 
adverse S/N ratios with the aid. I think the former portion will set reason- 
able target values for hearing aid assistance and the finer ccxnponent will 
differentiate among aids. 

Why don't I stop there so we can get some reactions? 



